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EXECUTIVE SUMMARY 


State report cards should help families, educators, policymakers, and other critical stakeholders in 
your state understand and act on education information for schools, school districts, and your state 
as a whole. While each state finds itself in a different place regarding its own progress in building 
state report cards with such an impact, emerging research and best practices can inform all states 
as they engage in continuous improvement for state report cards. 


This resource proposes considerations to state education leaders — key questions to ask along with 
emerging research and best practices that can inform the answers. The considerations encompass 
the following areas: 


e Feedback and engagement strategies, 
e Data and content, 


¢ Development and sustainability. 


Communicating Performance: A Best Practices Resource for Developing State Report Cards is 
intended to inform your next steps regardless of where your state is in the journey to build state 
report cards that deepen understanding and inform action among key stakeholder groups. It is 
also intended to help states build on progress made through implementation of the Every Student 
Succeeds Act (ESSA) to engage diverse stakeholders. 


This resource should help your state meet the following the goals: 


e Tie the state report card to your state’s theory of action to improve education 
outcomes by identifying priority users of the report card and the actions the report 
card should inform. 


¢ Evolve the way the state report card displays data by moving beyond data tables to 
visualizations that help tell stories that users will remember. 


¢ Build buy-in and use among users by deploying multiple engagement mechanisms 
such as advisory groups, town hall meetings, presentations, and communicating 
updates through the state’s website. 


¢ Improve users’ experience with and understanding of the data by gleaning specific 
feedback on current state report cards; draft “wireframes” that show how new 
report card pages could be structured; draft visualizations including text, colors, and 
pictograms; and potential elements of functionality. 


e Assure users that data included in the state report card are technically defensible and 
meet the highest standards of quality, and that privacy is safeguarded. 


¢ Sustain momentum to keep improving the state report card — to answer more and 
better questions, to build in functionality that users need, and to ensure the state has 
the resources and support to do so. 
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INTRODUCTION 


State leaders invest considerable time and resources in building and enhancing state report cards 
designed to help parents, educators, policymakers, and other critical stakeholders understand 
education information for schools, school districts, and for the state as a whole.' They do so in the 
context of their broader strategies for public reporting, communication, and engagement, which 
include numerous avenues for communicating with parents, educators, policymakers, and other 
critical stakeholders. Report cards should be one of the most accessible ways for such stakeholders 
to glean information to make good decisions. Providing information to the public on how all 
groups of students are performing academically and ensuring all groups of students have access 
to essential resources for learning are key tools for parents and community groups in making 
important decisions for their children. 


At a minimum, state report cards are meant to ensure basic transparency, providing high-quality 
education data and information to the public in an accessible format. Report cards inform 
families, educators, and policymakers of the progress a state, and schools within a state, has 
made toward the state’s goals. They reflect a state’s openness to communicate with credible 

data and information. Report cards should also go beyond this minimum aim of transparency, 
deepening understanding about the state’s public education system and informing actions that 
serve to improve the education of all students in the state. Report cards can help all stakeholders, 
especially families, understand what the data mean and why these data are valuable. 


In recent years, several states and partner organizations have been taking more intentional steps to 
find what it takes to build and enhance report cards that meet the dual purposes of transparency 
and informing actions. With their leadership, they have contributed to a growing research base, 
lessons learned, and exemplars to inform their own next steps as well as those of states just 
starting out. 


The purpose of this resource is to share emerging research and best practices with state 
education agency leaders, helping you think through key decisions in all phases of state report 
card development and continuous improvement. The considerations encompass feedback 

and engagement strategies, data and content, design and structure, and development and 
sustainability. They are intended to inform your next steps regardless of where your state is in 
the journey to build state report cards that deepen understanding and inform action among key 
stakeholder groups. This resource is intended to complement the State Guide to Building Online 


School Report Cards, along with other guidance documents for states. Please see the appendix for 


a full list of resources. 


1 This resource is focused on the public, online dynamic, or static “report cards” that state education agencies 
(SEAs) produce to communicate data and information to the public at the school, school district, and state levels. 
The best practices it contains may be applicable to other forms of public reporting. They may also be applicable 
to reporting that is not provided to the public, such as student-level report cards or assessment reports that are 
accessible by families and educators. 
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ESSA AND BEYOND: USING REPORT CARDS TO MEET STATE GOALS 


State leaders can harness opportunities to make dramatic progress in the effectiveness of state 
report cards as they implement the Every Student Succeeds Act (ESSA). Implementation can 
provide an opportunity for you to gain momentum or learn from and continue to act on the specific 
insights gleaned about effective stakeholder engagement and feedback through recent efforts to 
engage a wide variety of stakeholders through your state’s ESSA development processes. ESSA is 
a starting point — it should not restrict your commitment to improving your statewide systems of 
public reporting. 


You can learn from ways the public reporting field has evolved to make data indicators actionable, 
to design for understanding and use, and to optimize the functionality of report cards to help users 
access the information they need in the way that is most useful for them. As a result, state leaders 
will not only enhance transparency of information, but will better position state report cards as key 
levers in each state’s theory of action to meet ambitious education goals for all students. 


To do so, each state will need to identify within its theory of action, who uses the report cards, 
what actions they take as a result of the information they find, and how they need to see the 
information to maximize their understanding and use. Your state may tether the state report card 
to the overall theory of action driving the state’s accountability system, prioritizing a report card 
that simply communicates accountability indicators and results. You may prefer, however, to devise 
a theory of action for the state report card that goes well beyond this scope. For example, your 
state may prioritize building a report card designed to illuminate data and information about key 
transition points for students across their educational journeys, spurring school and community 
action to focus on strengthening these critical points to help more students succeed. It is essential 
to ensure that state leaders have a shared understanding of the theory of action before work to 
build the report card begins. 


Figure 1 provides an example of how a state may specify the primary users for the report card 

and the priority actions they hope will be taken by each user as a result of the information. In this 
example, the state has chosen to differentiate each aggregation level of the report card (state, 
school district [or LEA], and school) by the primary users for the information at each level. Following 
ongoing feedback and engagement with the identified groups, the state will be able to customize 
the data elements and display to meet the needs of each audience. 


This exercise is intended to make sure each level of the state report card has its “best fit” with the 
primary audience, but doing so should not lead to “information silos.” In education, audiences are 
not easily segmented. Individuals fit within multiple stakeholder groups. For example, the chair of 
a state legislature’s education committee may also be an educator and a parent. As well, we want 
to make sure different user groups can use the state report card to communicate with one another 
— teachers need access to the same information that families have, for instance, to facilitate strong 


communication. 
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Figure 1: How state report cards can support a state’s theory of action 


State 


State policy leaders 
(e.g., state board 
members, legislators, 
governor, SEA leaders) 


ie 


Communicate and rally support for state goals 
for student performance and equity 


. Evaluate progress to meet these state goals and 


celebrate progress that is made 


. Identify bright spots, emerging trends, and 


patterns such as gaps among student groups 


. Highlight areas needing improvement or 


targeted resources 


. Build public capacity to use education data 


School district (or LEA) 


1. Local policy leaders 
(e.g., school 
board, mayor, 
superintendent) 


2. Business, community, 
faith leaders 


N GO oo ff W 


. Evaluate progress to meet state and local goals 


. Identify bright spots and emerging trends, and 


develop the means to learn and share from 
districts experiencing success with student 
groups of concern 


. Prioritize local resources 

. Inform community supports and partnerships 
. Illuminate and address equity issues 

. Mobilize communities to improve education 


. Understand how a district is performing within 


the state and among districts with similar 
demographics and establish connections 


School 


1. Families 
2. Educators 


3. Community partners 


. Answer priority questions from families about 


their child’s education 


. Benchmark similar or nearby schools 


. Monitor progress to meet goals for overall 


improvement and equity aims 


. Plan school improvement efforts 


5. Help families contact and engage with the 


school 


Given advances in technology and heightened understanding about how to optimize user 


experience, states also have an opportunity to transform how information is communicated in the 


report cards. In recent years, the field has made great advancements in evolving beyond tables of 


data to charts that illuminate patterns and trends, and then to visualizations and infographics that 


present data in ways that are user-friendly and easy for stakeholders to understand. Ultimately, 


your state has an opportunity to drive understanding and use of information by using data, words, 


and pictures to tell stories (Figure 2). Stories have a structure — a beginning, middle, and end — that 


people are more likely to learn, remember, and share with others. Stories have the power to shift 


perceptions and prompt actions. 
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Figure 2: Evolution of data presentation 


Tables > Charts > Visualizations > Infographics > Telling stories through data 


Figure 3: Colorado fiscal reporting infographic 
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On average, 
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state, for a total of 
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Amended in 2014, the Colorado Public School Financial 
Transparency Act requires the Colorado Department of 
Education to publish a financial transparency website that 
makes user-friendly information about school and district 
expenditures available to the general public. The public 
website became available on June 30, 2017. In addition 
to providing visually-appealing expenditure (“spending”) 
information for schools, and revenue (“funding”) and 
expenditure information for districts, the site includes 
narratives from district leaders to give context for the school 
and district. The state also tells its own story through the 
fiscal reporting infographic in Figure 3 below. 


ESSA acknowledges this evolution, encouraging states 
to raise the bar in how they leverage report cards for 
educational improvement. The statute sets detailed 
requirements for state report cards (see appendix). Non- 
regulatory guidance issued by the U.S. Department of 


Education (USED) in January 2017 provides suggestions 


for both content and the way states engage with their 


residents.’ It directs states to engage a broad set of 
stakeholders in the report card development process, 
including those who may not traditionally have a seat at the 
table. It instructs states in ways to ensure accessibility of the 
information, from use of color to translations. The statute 
requires states to give users the ability to dive deeper 

into assessment data through cross-tabulations. Finally, 

it specifies not only a set of required data elements,* but 
descriptions to help users make sense of the policy context 
behind the information. 


2 The non-regulatory guidance that pertains to ESSA statutory language remains applicable, but portions 
that pertain to the accountability and State Plan final regulations affected by the resolution of disapproval from 


Congress are not applicable. 


3. ESSA-required data elements include 1) ESSA accountability indicators; 2) state and LEA data collected 
through the Civil Rights Data Collection (CRDC); 3) educator qualifications; 4) state performance on NAEP; 5) per- 
pupil expenditures; and 6) postsecondary enrollment rates by high school. 
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CONSIDERATIONS FOR STATES BUILDING AND ENHANCING 
REPORT CARDS 


FEEDBACK AND ENGAGEMENT 


When developing their state report card, states should first consider how they will lead with 
feedback and engagement as an initial and ongoing component of their report card development 
and continuous improvement cycle. Doing so will ensure that the report card is grounded in its 
purposes to ensure transparency as well as support strategic actions that lead to educational 
improvements. As states have learned through stakeholder engagement and feedback strategies 
in developing their ESSA plans, starting engagement early with specific opportunities for feedback 
builds buy-in and ownership by allowing stakeholders to be part of the process. Too often, 
feedback and engagement are sought only after there is a solid draft or prototype for potential 
users to review. While the instinct to wait until there is something to respond to is well-intentioned, 
it increases the likelihood that the feedback will be too narrow, missing “big picture” insights that 
should have been surfaced earlier in the process. Engaging users early in the process also creates 
opportunities for stakeholder ownership and buy-in. They will likely feel more compelled to use 
and spread the word about the tool if they felt involved in its creation. 


Ongoing feedback ensures that the report card focuses on its purpose and the needs of users, 
encouraging the full range of primary users to take informed action. Feedback from stakeholders? 
and users at all phases of development is critical to make sure the report card provides clear and 
accurate data, effective design and content, and the functionality users need. Clearly, feedback will 
illuminate how user needs for data, design, and functionality vary, and states will need to consider 
how to differentiate user experiences to meet those needs. In some cases, technology may make 

it possible to embed feedback tools into the report card, through back-end analytic reporting on 
user behavior, but also by encouraging users to comment on what they see. 


Engagement ensures that the report card will reach its intended audiences — and, in turn, helps 
deepen understanding of how to read and use the report card data. By building buy-in among key 
constituencies, it lays the foundation for sustainability of the state’s public reporting strategies. 

As such, states should engage early and often with state leadership (e.g. governor, legislature, 
state board of education), local district and school leaders, business and community leaders, and 
teacher, family, and student groups. 


As states craft feedback and engagement plans, they should consider the following questions: 
1. Who should be involved? The plan identifies all those who need to be involved in the process. 


Primary users: As outlined on page 3, the priority for states is to identify the primary users of the 
report cards for initial and ongoing feedback, such as families (who represent a broad range of 
perspectives), educators, administrators, and state and local policy leaders. In many cases, you will 


4 Please see appendix for a list of potential stakeholders within and external to the SEA. 
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need to build or strengthen new relationships to effectively engage individuals, especially those 
who have traditionally been disadvantaged or marginalized. It is critical that the engagement plan 
include outreach to existing partners and champions, like the PTA, but that it also extends beyond 
that to solicit feedback from those whose voices are not often heard or who don’t regularly engage 
in dialogue with the SEA. In its guidance, USED highlights the need for states to be intentional 
about engaging those who “traditionally may be left out.” As such, you may consider building 
partnerships with agencies or organizations in your state with reach and credibility with traditionally 
under-engaged groups. For example, in Louisiana, EdNavigators, a non-profit founded by a group 
of educators and families, is partnering with employers to reach and engage directly with families 
in New Orleans. It is also providing guidance so that they can make informed decisions about their 
children’s school experiences, including providing assistance in using data included from student 
and school-level report cards. It has also led family meetings on behalf of the Louisiana 
Department of Education to vet improvements in the state report card. 


¢ Champions: The second priority is to identify those The District of 
whose ongoing guidance and support are critical for 
buy-in. In some cases, these will also be primary users. 
It is likely, however, that some groups that may not rise 
to the top as primary users could be a high priority to 


Columbia found 
success in creating 
a formal reporting 


engage to ensure buy-in and sustainability. For example, partnership across 
Delaware has benefited from high-level leadership from agencies (District 
the governor's office on the redesign of its report card. of Columbia Public 
In other states, district leaders have stepped into the Schools, the 

role as champions for the state report card. Finally, some Gives atte Sista 


states have cultivated champions from across the SEA, 
as well as other state agencies (e.g., P-20 longitudinal 
data system staff, higher education coordinating body, 
workforce development, health and human services). District of Columbia 
Public Charter 
School Board) to 
develop and deploy 
its Equity Reports. 


Superintendent of 
Education, and the 


¢ Technical advisors: Your state’s technical advisory 
group for assessment and accountability, and other 
advisors with expertise in educational measurement, 
will also play a role in evaluating reports to ensure they 
are technically defensible. 


2. At what points should they be involved? State should create a transparent process and 
timeline for development, clearly highlighting opportunities for stakeholders to engage in 
the process. It is helpful for states to be explicit about windows of opportunity for feedback 
during the process. States should involve users and champions early and at multiple 
points in the development and continuous improvement cycle of the state report card. 


¢ Gathering requirements: There are many ways to start getting feedback early in the 
process to develop requirements. The team driving the project should be able to 
gather feedback through a series of facilitated sessions, and synthesize it into both 
technical and user requirements. Those can then be tested again, but the work of 
creating the requirements documentation does not happen with stakeholder groups. 
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States have an opportunity to begin user research as requirements are being gathered 
— for instance, they can work to get feedback on the current state report card site or 

other similar sites. As requirements can be about design, structure, data, content, and 
functionality, it will be important to differentiate which groups might be best suited to 
develop which type of requirement. North Carolina gathered requirements from state 


board of education members, legislators, and 
representatives from stakeholder groups 
including business, community, families, and 
educators. They worked with them in small 
groups with facilitators to get feedback to 
develop requirements for reporting measures. 


Design and content development: States are 
engaging families, educators, and other users 
in the process of designing and developing 
content for state report cards. New Mexico, to 
honor the state’s “fundamental belief that our 
families and taxpayers have a right to know how 
their children and their schools are doing,” led 
community meetings during the 2015-16 school 
year to learn about families’ understanding of 
report cards. As a result, the state simplified 
and clarified language in the report card and 
has begun a process to update the report 
cards to enhance understanding and usability. 
Michigan is in the process of developing a 
transparency dashboard, designed with families 
as the primary audience. The intent is to use 
both traditional and non-traditional indicators 
to provide a more comprehensive picture of 
schools. The development process has included 
parent focus groups and surveys to make sure 
that the data reported are meaningful and 
understandable to parents. 


Building, testing, and launch: Early and 
frequent engagement and feedback lay the 
groundwork for launch and outreach. An 
emerging best practice across states is to 

keep stakeholders informed throughout the 
development process. Delaware has a webpage 
with updates about its report card launch. States 
are also advancing ways to more deeply engage 
users and champions in the testing process prior 
to launch. In Georgia, state leaders select school 


The Council of Chief State School 
Officers, in partnership with 
national civil rights, advocacy 
groups and organizations that 
represent historically underserved 
communities, developed “Let's 
Get this Conversation Started,” 

a comprehensive guide to 


stakeholder engagement in June 
2016. The guide detailed 10 key 
steps to authentic, meaningful 
stakeholder engagement: 


Getting Started 
1. Clarify your goals 


2. Work with partner 
organizations to identify and 
engage with your stakeholders 


Engaging Stakeholders 
3. Speak to your audience 
4. Use multiple vehicles 


5. Identify your best 
ambassadors 


6. Ask for input before decisions 
are made, and use it 


7. Keep your materials simple 
and brief 


8. Communicate early and often 
9. Keep your team informed 
Sustaining Engagement 


10. Turn these new connections 
into long-term relationships 


districts to test new features of its report card. This strategy not only improves the 
functionality of the system, it broadens ownership. Delaware also engages primary 
users in testing prior to launch through an actual working website behind a firewall. 
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A report card prototype that Tembo developed with Learning Heroes includes 
functionality that allows users to submit comments directly into the prototype. In both 


examples, the building and testing processes are iterative. 


¢ Sustainability and enhancement: Feedback and engagement must continue far after 
launch. States can use a variety of mechanisms to create feedback loops with stakeholders 
to help continue improving the site to help it meet its goals (see below). It is also an 
important opportunity to build capacity among primary users at the state and local level. 


3. How should they be involved? The plan should identify a variety of mechanisms for feedback 
and engagement with careful attention to timing and location to maximize access and 


convenience for participants. 


¢ Selecting mechanisms: States have a number of options for mechanisms to use for 
feedback and engagement, and these mechanisms differ in their potential to heighten 
engagement and provide precise feedback. Figure 4 suggests how several common 
mechanisms vary according to their potential for feedback and engagement. These 
mechanisms also come with specific benefits and challenges. Some mechanisms will be 
great opportunities for feedback (1:1 interviews), but would not engage many people, 
while others (town halls) would engage many people, but would not be great for feedback. 
States may want to determine the most appropriate balance of these types of engagement 
mechanisms. Figure 5 addresses several of these, along with considerations for states that 
choose specific mechanisms. 


¢ Planning for access: For in-person mechanisms such as town halls and focus groups, states 
should take care to ensure that the time and location lend themselves to wide participation. 
For example, states should consider offering various times (day, evening, weekend) for 
family sessions, and offering child care, meals, and translation services. The sessions 
should also be held at a variety of geographic locations around the state and within large 
cities. As part of the development process that led to Ohio's new report card in 2015, 
state officials made 23 trips across the state to seek feedback on the Ohio School Report 
Card, bringing materials tailored to each site. Where possible, neutral, community-based 
locations should be selected to ensure all participants feel welcome. CCSSO’s Let's Keep 
This Conversation Going Guide offers an array of engagement strategies and successful 
state examples. All of the strategies represent ways that SEAs can transform their current 
approach to engagement into a series of impactful efforts to reach, engage and build 
lasting relationships with all stakeholders. 


e Preparing for success: To maximize their impact for the report card process, and ensure 
participants have positive experiences, state leaders should plan time to prepare fully for 
feedback and engagement sessions. For sessions seeking feedback, a clear protocol should 
be developed — and reviewed — in advance.° For sessions emphasizing engagement, it 
is essential to agree on the key messages and the most effective messengers to deliver 
them. Regardless of the primary purpose of the session, state leaders need to have a solid 
grounding in the context in which the opportunity will take place. What experiences have 
participants had to date with report cards? What differences can you anticipate will emerge 
across participants (e.g., families and educators often have different views on A-F grading 


5 See appendix for a sample focus group protocol. 
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for schools) and how can the feedback process show that all views are valued? What 
education issues or concerns® such as data privacy’ can you predict will arise - and how can 
the session be structured to acknowledge them at the outset? State leaders will also need 
to set expectations about why feedback is being solicited and how it will be used. 


Figure 4: Finding a good mix of mechanisms to maximize feedback and engagement 
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Figure 5: Selecting mechanisms for state report card feedback and engagement® 


Formal advisory 
group 


1. Members can be 
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members can serve 
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champions for the 
work 


3. Flexibility to design 
working meetings to 
maximize feedback 


1. Anticipating need 
to make decisions at 
a point of authority, 
even if the advisory 
group has not 
weighed in 


2. Ensuring that 


members adequately 
represent the range 
of users 


3. Making sure 


all participants 
understand their role 
and responsibilities 
and the ultimate 
goal of the work 


1. What norms/protocol will be used 


. How can the advisory group be 


. How will the role of the advisory 


. What is the responsibility of 


to make decisions when there are 
conflicts in feedback, priorities? 


clear about the primary users 
and their priority actions in order 
to focus discussion and inform 
decisions and priorities? 


group shift through phases of 
the development and continuous 
improvement process? 


participants to communicate 
directly with their constituencies? 


6 Focus group research from Learning Heroes suggests that parents put the highest priority on information 
about school safety and educator credentials and often see differences in performance across schools as a result 


of funding inequity. 


7 See Data Quality Campaign for resources on state policy to safeguard data privacy: https:// 


dataqualitycampaign.org/resource/key-elements-strengthening-state-laws-policies-pertaining-student-data-use- 
privacy-security-quidance-state-policymakers/. 


8 For more information about leveraging feedback and engagement strategies, see CCSSO’s 
Let’s Keep This Conversation Going guide: http://www.ccsso.org/Documents/2016/ESSA/ 


LetsKeeptheConversationGoingGuide11022016.pdf. 
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12 


Town halls 


town halls) 


(including online 


1. Likelihood of 


communicating 
priority messages 


. Potential to engage 


a broad set of 
stakeholders in one 
setting 


1. Ensuring diverse 


participation 


. Difficult to collect 


detailed feedback 


. Making sure 


all interested 
participants are 
aware of these 
meetings 


. Making sure these 


meetings are 
accessible to all who 
want to participate 


1. How can partnerships and other 
efforts to build relationships 
help you get the word out to 
participate? 


2. Who are the right messengers? 


3. How can the state record and 


share the town hall? Can the 
state host a “virtual” town hall for 
those who cannot travel? 


4. How can the state partner with an 


organization to host the meeting 
and encourage more individuals 
to attend? 


5. How can the state collect contact 


information of participants at 
these meetings and use that 
to inform them on how their 
feedback was used? 


Presentations 
at existing 
meetings, 
including 
those hosted 
by community 
organizations 


. Built-in audiences, 


often those needed 
to be champions 


. Ability to segment 


messages and ask for 
feedback by audience 


. Time limitations 


. Gaining focus 


and attention of 
participants if 
multiple topics are 
being discussed 

in addition to the 
report card 


1. How can states leverage existing 
meetings beyond the “usual 
suspects” to meetings where 
families and others from under- 
engaged groups will be? 


Focus groups 


. Feedback can be 


tailored and precise 
with open-ended 
questions, follow- 
ups to press for 
understanding 


. Small groups of 


participants can 
surface areas of 
agreement and 
disagreement 


. Getting the timing 


right — pulling 
together a focus 
group to answer 
near-term needs for 
the development 
process 


. Recruiting 


participants who 
reflect the priority 
users 


. Using a third- 


party facilitator so 
participants can 
feel open in sharing 
feedback 


1. How can focus groups be utilized 
prior to the data and design/ 
content phases (e.g., sharing 
existing reports)? 


2. What partnerships are needed 
to recruit a strong set of 
participants? 


3. How can you make sure focus 
groups include individuals 
representative of your state’s 
population? 
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Websites and 
online surveys 


1. Ability to reach 
stakeholders in a 
uniform, transparent 
way 


2. Additional 


information can be 
shared through the 
website and email 
correspondence with 
survey link 


1. Ensuring target 
audiences view 
website, respond to 
survey 


2. Inability to press for 


understanding 


3. Survey respondents 


are sometimes 
limited to certain 
demographics 


1. What existing websites, surveys, 


or email lists could be harnessed 
to glean feedback and reinforce 
messages? 


. How will results from surveys be 


shared and to whom? 


. How can you partner with groups, 


especially community-based 
organizations, to ensure good 
participation among diverse 
populations? 


. How can you make the survey 


available in multiple languages? 


. Can you ask for demographic 


information to track who is 
responding to the survey, and 
which audiences you may need 
to target further? 


1:1 interviews 


1. Depth of discussion 
can uncover 
important insights 


2. Talking to one 


individual may 
illuminate information 
that would not be 
shared in a group — 
such as disconnects 
between what is on 

a report card and 

the message a family 
member receives 
from a teacher 


1. Insights may not be 
generalizable 


2. Resource, time- 


intensive 


. Are there particular questions 


that require deeper 
conversations? 


. How can you identify individuals 


who will give you deeper insights 
necessary to move the work 
forward? 


User testing 


1. Observing how users 
actually interact with 
the data, design 
and content, and 
functionality 


2. Ability to target 


users with precision 
through services that 
specialize in user 
testing 


1. Resource, time- 
intensive 


. What high-priority questions 


should be asked to test site 
usability, or particular reporting 
concepts? 


. How can you make sure you 


are testing among users with 
different needs, such as limited 
English proficiency, literacy skills, 
or disabilities? 


Embedding 
questions ina 
live report card 


1. Open-ended 
questions, such as 
a “Notice anything 
confusing?” comment 
box, can pick up 
important issues that 
may not otherwise 
get raised. 


. How will ongoing advisory or 


governance groups for the report 
card address information coming 
through this mechanism? 
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DATA AND CONTENT 


Families and other users must have trust and confidence in the information in state, local, and 


school report cards. States must ensure that data in state report cards are of the highest quality, 


that privacy is protected, and that measures are timely, clearly defined, and consistent across 


different levels (e.g., student-level to school-level) and different reports. 


States should consider these questions in all phases of report card development and 


continuous improvement: 


9 


PARCC 


Percentage of students at each performance level on the PARCC assessments. 


1. What data and performance metrics will be reported? 


¢ Meeting minimum requirements: ESSA and other federal education laws prescribe a set 
of data and information that states must include on state report cards for schools, districts, 
and/or the state as a whole. In addition, many states have policies in statute or regulation 
that require additional data elements to be included in state report cards. Taken together, 
these requirements set a “floor” — the minimum set of data that a state must include. 


¢ Overall Grade &Demographics > 


SELECT ©) Composite ELA Mathematics 


COMPARE: 2016, 2015 - Grade3 - All Students 


= Ed 


Demographics Grade Year % of Students Achieving Performance Level 


All Students Grade 3 2016 


Grade 3 2015 


2. How do we ensure data used in the report card are of the highest quality? What is 
data quality? According to NCES, data quality entails accuracy, completeness, timeliness, 
validity, and consistency.’ 


¢ Building capacity at the local level: Given that most data are collected at the local 
level, it requires coordination and capacity-building from the school building to local 
education agency to state levels. Kansas has developed a Data Quality Certification 
program with specialized tracks for LEA data entry personnel, data coordinators, 
program staff and administrators. The purpose of the program is to build a culture of 


Institute for Education Sciences, May 2014, Data Quality: Striking a Balance. SLDS Issue Brief. 
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quality data in the State of Kansas to improve data quality and reduce the negative 
effects of inaccurate/missing data. Last year KSDE provided both in-person and 
webinar trainings to individuals in 284 out of 286 public school districts. 


Involving the right people at the right time in decisions: Collaboration is essential 
to developing high-quality data for public reporting. State leaders with responsibility 
for report cards must involve ClOs, data managers, policy staff, and other experts in 
decisions about data for report cards. 


Driving data use at all phases of development and continuous improvement: 

Data use and quality are a virtuous cycle — as more reports are developed and used 

to support and inform work to improve student performance, more people ask more 
questions of the data, leading to improvements in data quality. States can take steps 

to encourage use of data beginning in the gathering requirements phase, engaging 
primary users with real data on proposed measures. They can continue to use feedback 
and engagement opportunities throughout design and content development, and 
testing and launch. Questions that primary users pose in these phases can alert states 
to potential issues with data quality. 


Deploy quality assurance routines on data after each phase of development: 
Similarly, states would benefit from establishing formal quality assurance routines on 
data elements not just prior to launch, but throughout the process of development and 
continuous improvement. Given that data quality and use are mutually reinforcing, doing 
so across the cycle provides more opportunities to catch errors or misunderstandings. 
Routines must be based on data quality rules and data governance that support the 
rules, and should leverage a clear, easy to read data dictionary that includes information 
about how data are used, the frequency of data collections, and what data source is 
considered the truth if more than one collection is used for the report card. States may 
consider establishing an agency-wide Quality Assurance and Standards group with broad 
representation of data analysts and producers. 


3. How do we provide assurance that data on the report card protect privacy? 


Establishing and enforcing clear reporting rules to protect privacy: As states have 
transitioned to reporting aggregate data to the public based on databases of student- 
level data, the importance of ensuring data privacy has risen. Privacy is also a primary 
concern of families and policymakers. State data management leaders should determine 
policies that ensure consistent use of minimum n sizes, data suppression routines, and 
other strategies.'° Each policy decision will involve significant trade-offs. For example, 
higher n-sizes will improve privacy protections, but will decrease access to data. All 
decisions should be made with ongoing stakeholder engagement and communicated 
broadly. Quality assurance routines should check to make sure these policies are followed 
for all reported data. The District of Columbia Office of the State Superintendent of 
Education (OSSE) details its procedures for ensuring the privacy of data in accountability 
and public reporting in its ESSA plan (pages 10-11), including a minimum n-size of 10, 
suppressing individual outcome categories, secondary data suppression, and using top 
and bottom coding for extreme percentages. 


See Data Quality Campaign resources for state education agencies — Roadmap to Safeguarding Student Data 
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4. What steps can we take to increase the timeliness of data? 


¢ Updating report cards throughout the year: Report cards should be updated at least 
once a year — at a minimum, along with the release of accountability results. Every update 
becomes a built-in opportunity for proactive communications. Meanwhile, a range of 
data that answers important user questions is finalized throughout the year. States should 
consider opportunities to keep report cards fresh with updated data throughout the year. 


5. Are the measures defined clearly and consistently? 


¢ Simplifying measures: To support their theory of action for reporting, states should 
consider how to leverage their report cards to build users’ capacity to understand and 
use the data. Where possible, states should consider ways to simplify data definitions to 
encourage understanding and use. 


¢ Providing definitions near data: Where possible, states should consider how to report 
data using familiar terms and avoiding jargon (particularly for family-focused report cards). 
Learning Heroes has found a number of common words and phrases that have different 
meanings to families outside of the school or district than they do in the school context. 
For example, families often take school “climate” to pertain to temperature or weather, 
and “culture” to pertain to ethnicity or race. Learning Heroes has found that a phrase 
such as “learning environment” better communicates the intent. In some cases, clarity will 
require using technical terminology. States should find ways to provide data definitions 
in an accessible manner, through hover-over features or otherwise nearby the data that 
are reported. For complex data definitions, states should take opportunities to explain 
the data in engaging ways. For example, Georgia posts a video on its website to explain 
its growth model in easier to understand language. Virginia has an FAQ and Glossary of 
terms for their School Quality Profiles. 


Exemplar | District of Columbia Equity Reports (defining specialized education words “jargon”) 


ATTENDANCE 


All students benefit from a high in-seat attendance @ rate, or the average IN-SEAT ATTENDANCE RATE 


percentage of students in the classroom on a given day. Any absence, excused 


or unexcused, counts against this number. 8 9 8 ys 
. oO 


In-seat attendance rates divide the total number of students’ days present by the 


total number of students’ days enrolled in the city's schools. 


Students’ subgroup @ status is determined according to the rules outlined under 


Student Characteristics. 


¢ Providing narrative context near data: In some cases, feedback may point out areas 
where data elements promote misunderstanding or confusion rather than clarity. 
According to research from Learning Heroes, in these cases, providing narrative 
information that reinforces the context underlying the data can be helpful. 
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Exemplar: Illinois (providing context to data) 


Freshmen On Track BAS 
Percentage of grade 9 students who are on track to graduate 
® Year 2016 
View Trend 


I State (82.4%) 


82.4% 


Freshmen On Track 
In State 


Explanation of Display Resources 


Students identified as “on track” have earned at least five full-year course credits (10 semester credits) and have earned no more 
than one semester “F" in a core course (English, math, science, or social science). Course credits from summer sessions are not 
included in this calculation. Freshmen on track is a key predictor of high school success. Students who finish the ninth-grade year on 
track are almost four times as likely to graduate from high school as students who are not on track. Research shows that the 
number of students on track and the graduation rate rise when schools actively intervene by identifying freshmen at risk and 
providing tutoring, additional instruction, and other individualized services. 


e Ensuring consistency across student-level and aggregate reports: Given questions 
from users, Delaware has identified a significant need to ensure consistency of data 
from the student level to aggregate, public reporting. State leaders emphasize the 
need to be “100% in sync” across the two levels through effective data governance. In 
addition, states should be aware of where data elements are reported across different 
aggregate report cards — and ensure data consistency in these cases as well. 


6. Are there opportunities to streamline reporting? 


¢ Taking stock: States, along with standing advisory groups, should regularly take stock 
of reports that display similar data, even across agencies, to check for consistency of 
definitions and business rules. As it does so, Delaware asks whether similar reports are 
still needed, and identifies opportunities to streamline reporting. 


¢ Serving data where it is used the most: Some states report a wealth of data (e.g., 
reporting postsecondary enrollment information on higher education agency websites), 
but not in the school-level report cards where families and other users are accessing 
data most often."' States may consider reporting some of these data on the state 
report card, and including a link so users who want additional detail can find the more 
comprehensive data set. 


11. See Data Quality Campaign, A Time to Act 2017. 
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DESIGN AND STRUCTURE 


States that position report cards as a part of their overall strategy to improve education for 

all students will pay considerable attention to design and structure throughout all phases of 
the development process. Simply, families, teachers, policymakers will not take action based 
on information in report cards if they see information that is overwhelming, confusing, or 
unreadable. While states with sufficient capacity can evolve online state report cards to take 
advantage of new technology and user experience principles, even those with more limited 
resources can develop static report cards with far greater impact by taking time to engage with 
users to advance design and content. Effective design requires states to wrestle with a variety 
of trade-offs — for example, prioritizing among different user needs, balancing simplicity of both 
design and content with needs to provide opportunities to engage with data in more complex 
ways, and the like. It also requires a mindset open to learning and adapting based on new 
knowledge and understanding from users. 


States should consider the following questions throughout the report card development and 


continuous improvement process: 


1. Who uses the report card? Why and how are they using it? The most important aspect 
of good design is to understand who your users are, why they are there, and the best way 
to serve the content to them at the right level of granularity. 


e Identifying priority users: Very early in the process, states should clearly identify 
the priority users of the report card, and all decisions about design and functionality 
should ripple out from this one. Families will be a critical audience for all states. 

They may be the priority audience for report cards, particularly at the school level.’ 
Educators - teachers and administrators — are also critical users of aggregate 
information at the school and district levels. It is essential for states to ensure that 
educators have access to and capacity to interpret state report cards to help answer 
families’ questions and proactively frame report cards (such as through a cover letter) 
in the context of the school’s goals and strategies. Educators are also an important 
audience for aggregate report card information that provides opportunities to “drill 
down” to more complex or student-level information. As an example of a way a state 
uses multiple tools for reporting, Massachusetts provides extensive data for district 
and school leaders through its Excel-based District Analysis Review Tools (DART). 
Policymakers, business, and community leaders are also important users of aggregate 
information at the school, district, and state level. Finally, researchers are frequent 
users of report card information, so providing entry points for them to access files to 
download should be a consideration for all states. See Figure 1 for an example for 
how a state may articulate its priority users. 


12 Research from Learning Heroes indicates that parents are most interested in questions that concern their own 
child's schools. 


13 Please note that DART is a reporting tool used in Massachusetts but is not used as a report card. Additionally, 
please not that users can access online DART tools in School and District Profiles. An example is available here. 
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e Finding out why they use the report card: You can use feedback and engagement 
mechanisms to find out more about why families and other stakeholders use report 
cards. They will likely hear a variety of answers within any particular group, and can 
use this information to inform design, structure, and functionality. States can also use 
this insight to design online report cards that adapt based on answers to questions 
such as, “Why are you here today — to find a school in your community? Compare 
schools? Learn more about my child’s school?” States can also plan their report card 
continuous improvement processes to inform ongoing understanding of the reasons 
users access report cards, including embedding analytics in online report cards to 
show the types of information users are accessing most frequently. 


¢ Designing for how they access the report card: Given that more parents and 
educators are accessing online report cards through their phones, tablets, and other 
mobile devices, incorporating responsive design is becoming a far higher priority. 
States should also consider ways optimize load time, and avoid putting information 
behind tabs or requiring drop-down menus in the design process. States should 
leverage feedback and engagement mechanisms to test how parents and other users 
navigate through online report cards. Many users, meanwhile, want — or need - to 
access and print a static report such as downloading a .pdf file from a website or 
receiving a hard copy of a static report from a school or community organization. 
New Mexico has focused efforts on developing informative and engaging static 
reports at the school level tailored to parents. 


e Giving context: Without context, data are just numbers without any meaning. States 
should consider how to design report cards in such a way as to provide the context 
that users need to understand and act on the information. Showing how the data 
relate to benchmarks helps build context. In addition, providing the most recent 
assessment information for a school is important, but families also want to see the 
most recent data in context of trends or historical information to answer questions 
such as, “Is the school improving over time and by how much?” 


¢ Helping families use the report card in conversations with educators: State report 
cards could also provide families with suggested questions to ask when meeting with 
a teacher or administrator or information about who to contact with questions about 
the information in the report card. 


¢ Understanding what tools are already available: To avoid redundant reporting or 
reinventing the wheel, understand what local systems are reporting and determine 
how the state report card can add value. 


2. How should information be prioritized? As the active design work progresses, states 
should begin to make the tough decisions about how content will be structured to drive 
understanding and use. States and data designers can work together and engage with 
users through wireframes that show the structure for how information could be prioritized 
and organized on a specific page. 


14 For trend data in which calculation methods change over time, states should normalize the data to ensure the 
trend shows meaningful information. 
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Letting the story lead: As discussed earlier with the example from Colorado's 
financial transparency website, states are learning that people learn through 
stories, remember stories, and take action based on stories. When someone 
engages with a state report card, they should be able to weave a story from the 
data - a story with a beginning, such a compelling question, then data and context 
through brief narrative, and then a conclusion — a clear answer, a suggestion for 
action, or a direction to go for other questions or more detailed information. In 
some cases, however, gaps in data, changes in baselines, or other factors may 
present significant challenges. 


Preventing information overload — for users and staff: Many families and other 
users get overwhelmed when they see traditional state report cards that provide 
numerous data points in table format. They often won't find information they need, 
or worse — will leave frustrated or misinterpret what they see. States are learning 
how critical it is to give users the information they need the most to prevent 
information overload. States can start by stating the questions the data are meant 
to answer, what the data mean, and any benchmarks that help the user interpret 
the data. User research has shown the benefits of designing with the highest 
level, most important information at the top, then cascading information down 
with further details. It also shows the benefits of letting users “opt-in” to details 

- clicking ona link, scrolling down, or turning a page. Information overload is a 
risk not just for users, but for staff and others designing state report cards. Today, 
state agencies collect a great deal of data, not all of which needs to be included 
on report cards. Staff should be careful not to get lost in the data as they are 
designing, just as they do not want to let users get lost. 


Organizing information: Information presented in state report cards should be 
organized in a way that reinforces the decisions made for prioritization and story- 
telling. States are learning that they can organize report cards by compelling 
questions, or by domains of information, through a site map. Many states are 
exploring “dashboard” approaches that show a limited set of high-priority data 
indicators through simple visualizations, with opportunities to then drill-down to 
more details. 


Balancing a variety of needs: State agency leaders and users do not always 

have the same priorities for information, again reinforcing the need for early and 
frequent stakeholder engagement. State leaders often prioritize information that 
reinforces state goals and initiatives, while families tend to be most interested 

in information about school safety and educator credentials.'> With this 
understanding in mind, states should prioritize data that are important to reinforce 
state goals along with clear explanations for why the data are important. Where 
appropriate, it’s also important to clearly articulate how the information comes 
together to provide a summative score or rating. 


For more information please see Overview of Research on Effectively Engaging Parents 
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Exemplar | Ohio (dashboard organization) 


| GRADE Performance Index 
The Performance Index measures the test results of every student, not just 


those who score proficient or higher. There are six levels on the index and 
schools receive points for every student in each of these levels. The higher the 
achievement level, the more the points awarded in the school's index. This 
rewards schools and districts for improving the performance of all students, 
regardless of achievement level. 


Performance Index * Calculation Pie Chart Trend 
Achievement Pct of Points for Points 
Level Students this Level Received 
Advanced Plus | 0.4 x Ls = 0.5 
Advanced 12.1 Xx 1.2 = 14.6 
Accelerated 24.2 x re = 26.6 
Proficient 34.6 x 1.0 = 34.6 
Basic 19.2 x 0.6 = 115 
7 5 5 0/ Limited 9.5 x 0.3 = 2.9 
“ O Untested 0.0 x 00 = 00 
90.6 of a possible 120.0 90.6 
A= 90.0 - 100.0% 
B= 80.0 - 89.9% 
C= 70.0 - 79.9% 
D= 50.0 - 69.9% 
F= 0.0 -49.9% 


3. How can information be communicated through text? 


e¢ Refining language: Families, business and community leaders, and other users 
prefer simple, familiar words to give context to the data in report cards. Test 
language through focus groups and other engagement activities with families, 
educators, legislators, and others — do not assume that words and phrases have 
consistent meanings across stakeholder groups. Aim to keep the language clear 
and straightforward (e.g., avoid compound or complex sentences). If states must 
use specialized words, provide explanations within the narrative. Run report card 
language through text complexity tools'® and refine language until it reflects an 8th 
grade reading level (e.g., use an automatic readability checker and/or check the 
Lexile level). 


16 CCSSO has a Short Guide to quantitative text complexity tools here: htto://www.ccsso.org/Navigating Text 
Complexity/Learn the Ropes.html. 
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Exemplar | New Mexico (using narrative to explain meaning of indicators) 


School Grade Report Card Final Grade 


2016 —cenified A 
Union Elementary 


District: West Las Vegas Public Schools 
Grade Range: 2-5 Code: 68157 


This Schoo! Ml 
Statewide C Benchmark [ea] 


School Possible 
Current Standing Grade Points Points 
How did students perform in the most recent school year? What percent a 
of students are on grade level? Did students improve more or less than 
expected? B 26.63 40 


4. How can information be communicated through visualizations? Data visualizations transform 
data into information that drives understanding and use. Similar to strong writing, strong 
visualizations depend on numerous rounds of editing to include only those elements that best 
convey a story. Although the data visualization field is always pushing the envelope with 
innovative techniques, many of the most effective techniques are simply highly-refined versions 
of traditional tables or graphs. 


e Keeping visualizations consistent: Focus on the story with consistent visualizations 
— keeping visualization techniques, colors, language, directionality, and comparisons 
among school, district, and state level data consistent allows users to orient themselves 
and then focus on the story, rather than on understanding a variety of visualization 
types. When color is used to convey meaning, such as levels of performance where 
red means low and green means high, make sure 
the meaning will be helpful to what you are trying 
to convey, and keep those meanings consistent Data Visualization 
across visualizations. Sites to Follow 
(recommended by 


Jonathan Schwabish): 


e Learning best practices in data visualizations 
and continuing to learn: The field of data 
visualization is continuously innovating and 
learning about the techniques that drive 
understanding, and it pays off to dive into current 
best practices and stay up-to-date on new 


* flowingdata.com 


* policyviz.com/blog 


developments. Websites such as flowingdata. * visualizingdata.com 

com and data visualization pioneer Edward Tufte’s 

edwardtufte.com contain numerous examples * storytellingwithdata.com 
and resources. Digital journalists have been 

leading many of the innovations in the field of * helpmeviz.com > you can 
data visualization. Research on data visualization submit visualizations in 


from the Northeastern University School of 


progress to get feedback 
Journalism suggests that text is highly important 
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in data visualization, that users remember data presented as pictograms (e.g., icons 

or logos incorporated into visualizations), and that some redundancy of data helps 
users remember information.® The USED’s report card guidance cites Jonathan A. 
Schwabish’s article from 2014, An Economist's Guide to Visualizing Data, as a strong 
resource on current data visualization techniques. The 2016 NCES Forum Guide to 
Data Visualization is directed at SEAs and includes detailed guidance on displaying 
data in ways to support analysis of information and support communication, along with 
guidance on processes to use to develop visualizations that convey meaning to users. 


Exemplar | Illinois (data visualization) 


Achievement Gap Fa 
The persistent difference in academic performance between different ethnic and racial groups, income levels, gender, and 
special student groups 
GRADES Summary ~ 
SUBJECT Composite’ ELA® Mathematics 
COMPARE 
*” Low Income/Non Low Income Black/White Hispanic/White 
Black/Hispanic Male/Female IEP/Non IEP 
Summary - Mathematics - Low Income and Non Low Income 
-27 2 -28 
oi | 


5. How can the state report card promote broad accessibility? 


¢ Complying with accessibility standards: For nearly 20 years, SEAs and other public 
agencies have been required to comply with technical standards to ensure online 
accessibility for users with physical or cognitive considerations. NCES has published a 
best practice resource for education data content providers that outlines techniques 
for addressing the standards." It also addresses standards from the Web Content 
Accessibility Guidelines (WCAG) initiative. 


e Testing color schemes: Estimates suggest that up to 7 percent of the male population 
in the United States cannot distinguish between red and green. States can use guides, 
such as that developed by the Foundation for Excellence in Education, to select colors 
that are easily distinguishable.” 


e Sizing up text: Text needs to be at a size that it is easily readable — both in narrative or 
explanatory blocks and within visualizations. User testing has clearly shown that families 
and other users often struggle with the small text sizes in traditional state report cards. 
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¢ Providing multiple languages: Today, few states provide report cards and wrap- 
around materials such as cover letters in languages other than English. As states 
continue to prioritize equity, consider opportunities to provide families and 
community leaders information in a variety of languages. See Figure 6 for an example 
from North Carolina. 


¢ Putting the report card out front: Many state report cards are difficult for users to 
find." They should be easy to find on the state’s website, through searches, and on 
other sites where families or other stakeholders will be (and with the fewest required 
clicks as possible). Illinois (www. illinoisreportcard.com) has gone an extra step by 
using an easy-to-find URL. 


Figure 6: North Carolina (providing multiple languages) 


NORTH CAROLINA 
SCHOOL REPORT CARDS 


2015 - 2016 NC School Report Cards 


You may select below NC School Report Cards for school districts or individual schools. First select whether 
you want district level or school reports. You can narrow down the list using the dropdown boxes. Also you 
can type a name or phrase in the search box to show only those reports for school or district whose name 
contains this name or phrase. 


First Previous 2 3 4 5 Next Last 


guilford ® District Reports Year: 2016 ¥ 


School Reports 


Year Language District Name 
2016 English Guilford County Schools 
2016 Spanish Guilford County Schools 


6. What functionality does the report card need to support its primary users? 


e Learning from users: States will begin mapping their functionality requirements early 
in the stakeholder feedback process. Do families prefer to search for their child’s school 
by typing in the name of the school, selecting the district and then the school from 
lists, entering their address, selecting from a map, or other method? Do families want 
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to see a static .odf report or engage with interactive features? How might families 
prefer to “drill down” from higher level to more detailed data? Do educators want to 
be able to compare performance information for their school with that of schools with 
similar demographics for benchmarking purposes? State leaders should begin mapping 
out these questions — and engaging users to answer them through focus groups, 1:1 
interviews, and other mechanisms — early in the development process. 


DEVELOPMENT AND SUSTAINABILITY 


States will need to carefully design their report card development and continuous improvement 


processes to ensure the system supports decisions made about data and content, as well as design 


and structure. The State Guide to Building Online School Report Cards (see appendix) includes key 


steps and timelines for report card development: 


1. How will the state manage the development process? 


Outlining decision-making protocols: Although states will engage and solicit 
feedback from a wide variety of stakeholders, a sound development process — one with 
a high likelihood of producing report cards on time and within budget — will identify a 
project team with an individual leader responsible for final approval of decisions. The 
project team itself may be comprised of different groups — for instance, a policy group, 
a technical (e.g., data and accountability) group, and a communications group. A sound 
process will also include formal change and release management protocols. 


Documenting requirements: It is essential that states formalize a set of data and 
content, design, and functionality requirements prior to the active design work. 
Not doing so risks significant production delays due to change orders — along with 
commensurate cost implications. 


Clarifying roles and responsibilities: States may work with a vendor or multiple 
vendors throughout the development process. Establishing clear roles and 
responsibilities, timelines, and review procedures with vendors will be critical for 
success of the project. 


2. How will the state plan for continuous improvement of the report card? 


Learning what works - and what doesn’t: In the building and testing phase, states 
can build an alpha site with limited functionality (even if the data and design are not 
quite right) to begin testing, followed by a beta version with complete functionality, and 
then quality assurance testing. After the launch, states can use analytics to find out how 
users are interacting with the report cards and identify needs to adapt functionality 
based on the metrics — both traditional metrics, such as the number of unique visitors 
to the site, repeat visitors, unique downloads, social sharing, mobile vs. desktop 

device usage, as well as metrics specific to state report cards, such as the popularity 

of particular school and district report cards and geographic areas requested in search 
and interaction with specific data exhibits. 


Planning for continuous improvement: Your state may consider planning to release a 
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version 1.0 of the report card, with expectations to answer new questions over time and 
enhance functionality in later releases. See below for an example from Illinois on sharing 
new functionality to compare schools and answering new questions about college-level 
coursework in high school. You may also consider adding wrap-around materials such 
as explanatory videos and cover letters to families and other stakeholder groups. 


¢ Keeping track of requests: Georgia maintains a running list of enhancement requests 
that are regularly reviewed and prioritized by the report card governance group. 


Illinois: Communicating Updates on Front Page of Report Card Site 


What's New 


June 22, 2017 
Compare Schools Feature Now Enhanced 


December 22, 2016 
Early College Coursework Information Now Available 


October 31, 2016 
Home Page Update 


More Updates 
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CONCLUSION 


Today, your state can take the next steps to weave data stories for families, educators, policymakers, 
and others that go well beyond basic public reporting transparency aims to support actions that 
improve education for all students. States can signal their commitment to continuous improvement 
by leveraging meaningful feedback loops and harnessing improvements in technology and design. 
They can do so with these users as partners in the process, inviting early guidance and building 
ownership along the way. Ultimately, states can see their state, local, and school report cards as an 
avenue for ongoing communication with those whose work directly affects students. They can use 
this feedback loop not only to enhance the report cards over time, but to support overall strategies 
to build capacity among families, educators, and policymakers to use education data to make 
informed decisions. States have much to learn from each other as they travel their own journeys to 
make report cards an integral part of their state's theory of action to improve student achievement, 
and will benefit from learning communities to share effective practices — and lessons learned. 
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APPENDIX 


State Guide to Building Online School Report Cards 


A State Guide to Building Paces 
Online School Report Cards [BAAR 


m Step-by-step process for 


A 12-month, development to launch guide to creating a designing highiqualtyireport 
next-generation school accountability reporting tool under ESSA card: 


O O O O 2 m Process organized by four 


streams of interconnected work: 


The Every Student Succeeds Act (ESSA) directs states to create report cards that are easy for parents and the public to access 
and understand. Additionally, states are required to provide more school-level data than ever before by highlighting multiple Feedback and Engagement; 


indicators of student performance, funding, teacher quality, post-secondary success, and more all in a parent-friendly format. Data: Desi gn an dl Gentian 


The timeline below outlines the necessary steps toward creating an ESSA-compliant report card (a full list of ESSA reporting Functionality and Development 


requirements can be found here) that is accessible and quality within a 12-month period. This timeline integrates four 
streams of interconnected work: feedback and engagement; design and content; data; and functionality and development. a Resources to guide your Wie) a, 
Resources are provided below on each stream to look to for advice and information during the development process. 

: . . . a = Glossary to define and clarify 
The streams of work are presented in four phases: gathering of requirements; design and content development; building, 
testing and launch; and sustainability and enhancement. Effective project management is critical for the development technical terminology. 
of a high-quality report card, and no one team within a state agency can do the work alone. These interconnected streams 
of work require close coordination of a robust project team of data experts, designers, developers, policy and communications 
experts, parents and community stakeholders. 


F : DATA QUALITY 
Counall of Giles Stete Schioc! Uficers CAMPAIGN 


2 i National 
CCSSO COLLABG)RATIVE DAC @ eExcelinEd LS PTA 


everychild. one voice” 
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O 
O 
O 


Phase |: Gathering of Requirements +—_@ 
Months 1-3 


Feedback and Engagement 


1 Identify your user audiences. 


[Look to local pilots that can serve 
as frameworks for a state platform. 


1 Conduct focus groups, deliver surveys, 
and convene your user audiences 
to understand what audiences 
expect, gain credibility with local 
constituencies, and build institutional 
knowledge. 


Announce your intention and goals 
for the development of new online 
school report cards; plan and share 
opportunities for engagement. 


Coordinating Project Partners 
Communications Staff 


Data 


[Determine the scope of the tool 
(local or statewide). 


-) Assess and document the quality 
and availability of your accountability 
and other data. 


[7 Review the SEA's accountability 
plan and other data that is required 
or desired on new report cards, data 
collection methodology, and 
anticipated data release schedule. 


Decide the high level categories and 
specific data indicators to be included 
and create a data catalog. 


Format the data and document the 
business rules for all data indicators 
that will be included in the tool. 


Resources 
Learning Heroes evidence-based parent 
tools and language. 


Learning Heroes, CCSSO, and PTA's 
Guidelines for SEAs Engaging Parents 


Coordinating Project Partners 
Policy Team 

Data and Accountability Team 
Communications Staff 


Resources 
DQC's Opportunity to Make Data Work 
for Students in the Every Student Succeeds 


Act. 


ECS's State Information Request: ESSA 
Accountability System Requirements 


Communicating Performance: A Best Practices Resource for Developing State Report Cards 


Design and Content 


© Apply knowledge about user audiences 
to establish criteria for the look and 
feel of parent-friendly report cards. 


(Use insights from focus groups and 
interviews to determine the tone 
and level of content that aligns with 
user expectations. 


[Begin to prioritize information and 
develop user journeys through the 
data and content that will be offered 
in the tool. 


Coordinating Project Partners 
Communications Team 


Resources 


ExcelinEd's My School Information 
Design Challenge findings 


DQC's Opportunity to Make Data Work 
for Students in the Every Student 
Succeeds Act. 


Functionality and Development 


- Determine website hosting parameters 
and select an open source content 
management system (CMS) such as 
Drupal or WordPress. 


[Determine and document the structure 
and workflow for the Application 
Program Interface (API) and how it 
integrates with the curation, cleaning, 
verification and uploading of data into 
the tool. 


[Create detailed technical 
requirements documenting the 
scope and functionality of the tool, 
such as sort, search, comparison 
and customization features, and 
presentation in multiple languages. 


Coordinating Project Partners 
Data and Accountability Team 
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Phase II: Design & Content Development 


Months 4-6 


— @ 


O 
o 


=} 


Feedback and Engagement 


X Develop a strategy for engaging 
local parents, educators, and 
community members through the 
design, development, and launch of 
the tool. 


Identify national, state, and local 
stakeholders to support the use of 
the tool and determine how to partner 
with them. 


OTest sample content and design 
concepts with user audiences and 
refine them based on feedback. 


Coordinating Project Partners 
Communications Staff 


Resources 

CCSSO's Let's Get This Conversation 
Started: A Guide on Stakeholder 
Engagement and Outreach 


CCSSO's Let's Keep This Conversation 


Going 


WW 


Data 


OConfirm different cuts at the data 
across years (over time) or subgroups. 


O Verify that all original data is “clean” 
and reconcile any irregularities or 
incomplete entries. 


Coordinating Project Partners 
Policy Team 
Data and Accountability Team 


Design and Content 


Ci Map data catalog to specific visualizations 
and content needs within the tool. 


Develop full color design mock ups 
showing the look of the tool on 
desktop and mobile. 


Determine data visualizations (charts, 
graphs) and content needed to support 
each data indicator in the tool. 


0 Create parent-friendly content explaining 
and supporting data displays. 


O Build messages that contextualize 
the data and empower parents to 
engage with their school. 


Oldentify partners, like district, school, 
and community members who can 
amplify the messages and promote 
the report card launch. 


Coordinating Project Partners 
Policy Team 

Communications Staff 

Data and Accountability Team 


Resources 


ExcelinEd's My School Information 
Design Challenge findings 


DQC's Opportunity to Make Data Work for 
Students in the Every Student Succeeds Act. 


| |p 


Functionality and Development 


O Finalize user experience journeys and 
create a wireframe and sitemap 
showing overall tool structure. 


O Set up data repository and establish 
the Application Program Interface 
(API). 


CO Begin development of the tool, including 
analytics tracking. 
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Phase III: Building, Testing & Launch 


Months 7-9 


=} 


Feedback and Engagement 


Invite users to engage with beta tool 
prior to public launch. 


OLaunch and promote use of the live 
tool. 


C1 Engage stakeholders around the live 
tool to get feedback on design and 
content ideas. 


Coordinating Project Partners 
Communications Staff 
Policy Team 


Resources 

CCSSO's Let's Get This Conversation 
Started: A Guide on Stakeholder 
Engagement and Outreach 


CCSSO's Let's Keep This Conversation 


Going 


ECS's Collaborative Stakeholder 
Engagement 


Ww 


Data 


OLoad or migrate data into the 
Application Program Interface (API). 


O Create a plan to accommodate newly 
released data during development. 


Design and Content 


O Finalize all aspects of the design of 
the tool. 


O Create launch messaging. 


Coordinating Project Partners 
Data and Accountability Team 


O Design and produce promotional 
collateral (one-pager, video tutorial, 
announcement, etc). 


Coordinating Project Partners 
Communications Staff 
Policy Team 


0 Develop the front end design and 


O Engage in rigorous quality assurance 


Functionality and Development 


underlying back end code needed 

to connect the data in the Application 
Program Interface (API) to what users 
see in the tool. 


testing to ensure all data and 
functionality are working correctly in 
the tool across environments and 
website browsers. 
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Phase IV: Sustainability & Enhancement 


Month 10 and Beyond 


a ®, 


O @ 


=} 


Feedback and Engagement 


O Continuously promote the report card 
and use it as a tool to reinforce state 
priorities; use new data releases as an 
opportunity to drive users to the site. 


O Thoughtfully review analytics to 
understand how the tool is being 
used; apply learning to enhancement 
of the tool. 


O Continue to engage with stakeholders 
upon new releases to test 
enhancements. 


Coordinating Project Partners 
Communications Staff 


Data 


O Create a calendar of projected future 
data for implementation in the tool. 


O Enhance the Application Program 
Interface (API) to accommodate new 
data indicators. 


Update existing data. 


Coordinating Project Partners 
Data and Accountability Team 


Design and Content 


O Enhance how data is prioritized, 
displayed, and explained based upon 
user feedback. 


O Create mock ups for new data 
indicators and changes to data 
visualizations required to 
accommodate additional years of 
data. 


Coordinating Project Partners 
Communications Staff 

Policy Team 

Data and Accountability Team 


is 7 ig | 


Functionality and Development 


O Develop new data displays and add 
or update supporting content. 


O Execute ongoing hosting and 
maintenance of the tool, such as 
updating the Content Management 
System (CMS). 


OTrain staff on use of the Content 
Management System (CMS) and 
data workflow. 


O Create an annual sustainability plan 
for future costs to keep the tool live 
and up to date. 


Policy Team Resources Coordinating Project Partners 
ExcelinEd's My School Information Policy Team 
Design Challenge findings 
DQC's Opportunity to Make Data Work Resources 
for Students in the Every Student CCSSO's Memo on State Report Card 
Succeeds Act. Requirements 
ExcelinEd's Know Your School Project 
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Glossary 


Data, design and development terms commonly used in online report card projects. 


Application Program Interface (API): An API is a set of instructions for accessing 

data in a database. APIs provide standardized “building blocks" of data that can be used to 
build many different types of websites. The use of APIs also makes combining multiple data 
points easier, exponentially increasing the value of those data sets. For example, Google 
Maps data is connected to numerous apps and websites via APIs that allow those apps and 
websites to combine their data with the Google Maps data. 


Front End Design: Think of a website or web tool as having two sides. The front end 
design is what a user or visitor sees. A front end designer makes deliberate decisions, 
informed by user experience (UX) best practices. These best practices inform where on the 
page information is displayed, how users navigate from one part of the web page or tool to 
another, and how information is prioritized. 


Back End Code: Think of a website or web tool as having two sides. The back end code is 
the engine of an online tool or website. This code does all the unseen work of accessing the 
API, inserting the data into a page and showing that page to the user on the front end. 


Beta Tool: A beta tool is like a first draft. Once front end design and back end code are 
complete, developers test a beta version of a website or tool. The website is not complete, 
and bugs are to be expected, but the point of a beta is to ensure that the user experience 
aligns to user expectations. Developing and testing a beta tool is a development best practice. 


Content Management System (CMS): A CMS provides a human-friendly way to 
manage the content that goes into a website, such as text, photos, video, etc. By creating a 
system that can manage this content, a CMS enables non-developers to update, delete and 
create new website pages and easily add content throughout the site. 


Data Catalog: The written documentation and inventory of the actual data points available 
to power an API, and how developers can access them, is called a data catalog. A quality 
data catalog includes business rules that define where data comes from, parameters regarding 
its content, and how it is computed. 


Business Rules: The business rules are the guidelines that describe proper use of data, 
including when certain data should and shouldn't be used, how data points interact and 
when data is considered out of date. 


Data Repository: A data repository is a collection of multiple, but related databases 
accessed by an API used to power a website. 


Sitemap: A sitemap is a visual flow chart showing the structure of how information will be 
organized on a website. A sitemap allows developers and designers to work from a 
common understanding of which pages link from a main page, which pages link from a 
subpage, and so on. 


Wireframe: A wireframe is a visual representation of how a single web page type will be 
organized. For most websites, three to five wireframes are developed for various page types, 
such as the homepage, a news page or other interior page. Wireframes are turned into 
templates by the front end designers, and used repeatedly throughout the site. 
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ESSA PUBLIC REPORTING REQUIREMENTS 


Section 111(h)(1)(C) of the Every Student Succeeds Act (ESSA) requires that every state “prepare 


and disseminate widely to the public” an annual report card. It directs states to ensure the report 


card is concise, presented in an understandable and user-friendly format, developed along with 


parents and presented “to the extent practicable” in a language parents understand. The law 


prescribes a number of “minimum” requirements for data and descriptions that should be included 


in the state report card. Note: data elements that include the parentheses (cross-tabulation) must 


be provided in such a way as to allow cross-tabulation at the state level by race/ethnicity, gender, 


English learner and not English learner, and students with and without disabilities. 


Descriptions 


The state’s accountability system, including 


o Minimum “n size” for student subgroups 


o Long-term goals and interim measures of progress for all students and each 
student group 


o Annual indicators 


o System for differentiating school performance, including the weights assigned 
to each indicator, the methodology for identifying a school as consistently 
underperforming for any group of students, and the methodology for identifying a 
school for comprehensive support and improvement 


Names and number of schools identified for comprehensive support and improvement 
and names and number of schools identified for targeted support and improvement 


Exit criteria for schools identified for comprehensive support and improvement, and for 
targeted support and improvement 


Data Requirements 


Academic achievement of all students and each student group — including homeless, 
military dependent, and foster-care students — for each assessment and at each level of 
achievement (cross-tabulation) 


Performance on the “other academic indicator” for elementary and middle schools, and 
the four-year adjusted cohort high school graduation rate for all students, all student 
subgroups, and homeless and foster care students (cross-tabulation) 


Number and percentage of English learners attaining English language proficiency 


Performance on the “other indicator or indicators of school quality and student 
success” for all students and each student subgroup 


Progress toward meeting the state’s long-term goals and interim measures of progress 
for all students and each student group 
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Percentage of students assessed and not assessed, for all students and each student 
group (including racial and ethnic group, economically disadvantaged and not 
economically disadvantaged, students with and without disabilities, English learners, 
male and female, migrant students) (cross-tabulation) 


For the state and each LEA, the measures of school quality, climate, and safety required 
by the Office of Civil Rights Data Collection, number and percentage of students 
enrolled in preschool programs, the number and percentage of students enrolled 

in accelerated coursework to earn postsecondary credit and in dual- or concurrent 
enrollment programs 


Teacher qualifications, disaggregated by high-poverty and low-poverty schools, 
including the number and percentage of inexperienced teachers, principals, and other 
school leaders; teachers teaching with emergency or provisional credentials; and 
teachers teaching outside of the subject or field of certification 


Per-pupil expenditures of federal, state, and local funds for each LEA and school, 
including personnel and non-personnel expenditures 


Number and percentage of students with severe cognitive disabilities who take the 
alternate assessment, by grade and subject 


NAEP results for mathematics and reading for 4th and 8th for the state and as 
compared to the national averages 


Postsecondary enrollment cohort rate for each high school, where available, for 
enrollment in public postsecondary institutions in the state, private postsecondary 
institutions inside the state, or postsecondary institutions outside the state 
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QUESTIONS TO CONSIDER 


Feedback and Engagement 


1. 


Who should be involved? 


2. At what points should they be involved? 


3. 


How should they be involved? 


Data and Content 


1. 


2. 


What data and performance metrics will be reported? 

How do we ensure data used in the report card are of the highest quality? 
How do we provide assurance that data on the report card protect privacy? 
What steps can we take to increase timeliness of data? 

Are the measures defined clearly and consistently? 


Are there opportunities to streamline reporting? 


Design and Structure 


1. 


2. 


Who uses the report card? Why and how are they using it? 
How should information be prioritized? 

How can information be communicated through text? 

How can information be communicated through visualizations? 
How can the design and content promote accessibility? 


What functionality does the report card need to support its primary users? 


Development and Sustainability 


1. 


2, 


How will the state manage the development process? 


How will the state plan for continuous improvement of the report card? 
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INTERNAL SEA COLLABORATION 


Internal SEA 


Executive office: As with any SEA initiative, the vision of the executive office or team should 
be reflected in the purpose of the state report card. Key considerations: Why is the report 
card important? What value does it bring to improvement efforts? How does it align with the 
overall vision of successful schools? By clearly articulating the answers to other teams within 
and outside the SEA, the communications team can help maintain alignment to the executive 


office's vision. 


Data information office: The information office will be central to the report card effort. They 
will need to be consulted to ensure the business rules are complete, data flow is clear, sources 
of relevant data are identified, and the appropriate representatives are included in data 
transfer and reporting efforts. 


Assessment: The assessment office is responsible for statewide summative assessment 
data that serve as the foundation for much of the state report card. This team will likely 
own the process for selecting, calculating, and communicating the assessment results. The 
assessment team will work closely with others in the SEA to ensure internal and external 
stakeholders understand the data, how it’s applied to other initiatives, and how to best use 
assessment data. 


Accountability: For those SEAs that have accountability and assessment teams in the same 
office, there should be a natural partnership between team members. In places where the 
offices are distinct, SEAs should foster a close interaction to promote policies and business 
rules that account for the nuances of the assessment. 


Standards/instruction/curriculum: The standards/instruction/curriculum teams understand 
the techniques that are effective in instructional efforts when using assessment and 
accountability information. It is important to involve these teams early and often so they can 
help reflect the perspective of local agencies and schools and serve as an ally for the SEA 
when contacted by districts and schools. Furthermore, they can help develop resources to 
facilitate the use of report card data for selecting instructional improvement strategies at the 
school and district levels. 


School improvement/federal programs: These teams will serve a vital role in the ESSA 
accountability system. Because they help clarify the system of supports and interventions 
for local agencies and schools, they will need to have a thorough understanding of the data 
used for accountability determinations and how parents and other vital stakeholders can use 
the data. 
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SAMPLE PARENT FOCUS GROUP PROTOCOL 


This is a sample. If it is helpful, we encourage you to adapt it to reflect your state’s context 
and needs. 


Context for Parents: Before you begin to engage parents in a conversation, be sure to set the 
context for why you are asking questions and how you plan to use their feedback. 


Introduction: Every year, our state generates school-level report cards to illustrate performance. 
Our state is committed to sharing information with parents in a timely and easy to use manner. We 
are interested in your feedback on how to improve our state’s report card. 


In addition to our state's commitment, in late 2015, a new education law was passed by Congress 
known as the Every Student Succeeds Act or ESSA. This law replaces No Child Left Behind and gives 
states more control and power over their school systems. The goal of ESSA is to make sure that every 
student in the state has the opportunity to receive a high-quality education. It calls for much of the 
same work that schools are already doing, such as holding all students to high standards and using 
quality tests to better understand where students need to improve. ESSA also requires that states 
publicly share how every school and district in the state performs on an annual report card. 


Opening question: In the month of X, our state releases these report cards. Do you remember 
receiving this report card or information about it? 


IF NO: 
e¢ Do you remember ever receiving information about the report card? 


¢ What do you know - or have you heard — about the report card? 


IF YES: 
¢ How did you receive the reports? When did you receive them? 


¢ Were you given any information about how to read or use the report cards? If so, was 
this information useful? 


¢ What was the first thing you did once you read the report card? 
¢ What did you look for? Were you able to find it? 


¢ Did reading the report card prompt you to think of additional questions? Did you know 
who to speak with to get those questions answered? If yes, did you reach out to anyone 
with those questions? 


¢ Was there any information not included in the report card that you wish had been included? 


e After you first read the report card, did you find yourself coming back to it at a later 
time? If so, for what purpose? 
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Distribute a copy of the report card to each parent. 


Framing: Please spend the next 7 minutes reviewing the report card individually. We will then have 


a group discussion. 


What are three things that stood out to you? 

What about the report card did you find most appealing? Why? 
What about the report card did you find the most confusing? Why? 
Is the report card missing any information? 

Is there any irrelevant information displayed? 


Does the order of the information help or does it make it more challenging to read the 
report card? 


Do the charts/graphics help you understand the data or do they make it more 
challenging to read the report card? 


Closing Discussion: 


Is there any other kind of information that you would want to see on the report card 
that we have not yet discussed? 


How would you like to receive the report card? (i.e., direct mail, e-mail, the backpack 
channel, in-person during parent/teacher conference) 


Would you rather see a collection of different kinds of data on your child’s school? Or 
an overall rating, like a number or letter grade, so that you could easily rank schools 
across the state? Or a combination of the two? 


How might our state better use our report card to call attention to places that are 
making tremendous improvement and/or excelling? How might our state use the report 
card to call attention to critical equity gaps? 


Closing Reminder: Before you adjourn, be sure to thank participants and remind them how 
you plan to use their feedback. 
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National 


CCSso P7A 


Council of Chief State School Officers 


GUIDELINES FOR EFFECTIVELY ENGAGING PARENTS IN FEEDBACK SESSIONS 


everychild. one voice.” 


The Every Student Succeeds Act (ESSA) presents a unique opportunity for state education 
agencies (SEAs) to meaningfully engage a variety of stakeholders in the development of their 
state and school-based report cards. Parents are one stakeholder group in particular that SEAs 
have traditionally faced a challenged in reaching and connecting with in effective and meaningful 
ways. One of the most common and effective ways of obtaining parent input is through informal 
feedback sessions. In doing so, there are important guidelines to remember to make the most of 
parent feedback and to foster sustained, trusting relationships. 


The following recommendations build on the Parent Engagement Guidelines for SEAs and 


Discussion Framework that CCSSO developed with the National PTA and Learning Heroes in 


November 2016 to support SEAs in better engaging parents as stakeholders in the development 
of state ESSA plans. These recommendations are based on qualitative research conducted 

across twenty states and multiple national surveys. It identifies effective strategies for in-person 
feedback sessions and is designed to help states make the most of these efforts. For additional 
parent-friendly resources, including FAQs, webinars, and supplemental materials, visit http://www. 
bealearninghero.org or https://www.pta.org/essastate. 


Vv Include a diverse group of parents. 


e You should ensure you have a diverse set of parents in your feedback session, as 
there are often different priorities, concerns and opinions among them. It’s unrealistic 
to expect that you will reach a scientifically representative group of parents, but you 
should try your best to reach as many demographics as possible. You should consider 
the following characteristics: 

o Race and ethnicity 
o Education levels 


o Income levels 


o Language (you should hold sessions for Spanish speaking parents) 


e Hold sessions in various areas of the state so you can reach urban, suburban and rural 
parents. Depending on the size and population of your state, you should try to have a 
minimum of three focus groups that cover different geographies. 


¢ To have a meaningful, manageable discussion, try to keep your session between ten and 
thirty parents. As groups get larger, it’s more difficult to get input from everyone present. 
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For historically hard to reach populations (including Spanish speaking, low-income, etc.), 
consider partnering with community or religious organizations, or parent organizations 
such as the PTA, that have trusted and established relationships with these parents. 
They can be integral messengers to share information, encourage participation and co- 
lead sessions. 


v Make it convenient. 


In the literal sense—meet parents where they are! Consider parents in various 
communities across the state and which locations would be the most convenient for 
feedback sessions. If a school is not centrally located, consider a location that is, and is 
also easily accessible via public transportation. 


Hold sessions at different times of day for parents who work different shifts or hold 
multiple jobs. In addition to the traditional evening meetings, offer a morning or 
early afternoon session. You may even want to ask parents first what time works 
best for them. 


Record or livestream in-person meetings and share the link via text and email for 
parents who are unable to attend. 


Think about amenities parents may need during sessions. Will they need childcare? If it 
is being held during a meal time, can you serve refreshments? Offering both of these 
tend to increase parent participation. 


v Meet parents where they are. 


Gauge parents’ understanding and perceptions of report cards in order to clarify any 
disparities or address knowledge gaps in language they understand. Information should 
be shared in a basic and factual manner. If it is not, it may be perceived as propaganda 
and therefore not trustworthy. Make every effort to be as specific as possible. 


v¥ Leverage effective communication channels. 


Parents’ preferred communications channels are email and text. If districts, schools, 
or other community organizations have existing list serves, leverage them to share 
information about the opportunity to attend a feedback session. 


Text messaging can be a useful method in confirming, reminding, and follow up 
with participants. 


In addition to email and text, you may also consider posting information on a school/ 
district or PTA website, using social media, and sending hard copies home through the 
mail and/or the backpack channel. 


Consider partnering with community-based organizations that would be willing to 
lead feedback sessions. In many states, these organizations have more established 
relationships with parents and may be viewed more positively than the state 
education agency. 
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¥ Make the content relevant. 


In an effort to create demand for the school level report cards, provide context and 
explain why they are important for parents—to give them information about how their 
child's school is performing and where improvement is needed. 


Talk about the report cards as a tool they can use to make more informed decisions 
and advocate for their child. Parents can use school level report cards much in the 
same manner they use student report cards: to measure progress and course-correct 
when necessary. 


Parents do not prioritize the same data policymakers and leaders do. 


They are more interested in information that directly impacts their child and they 
want multiple measures. To some level, they find every type of information useful. 
Ask questions to understand how parents prioritize measures, and you may want to 
consider providing a bank of measures for them to choose from. 


Ask questions designed to provide you with an evidence-base of the following: 


o Information parents want included on a school report card; 
o How they want to access the information; 
o How they will use the school report card; and 


o What language they do and do not understand. 


v Use evidence-based parent-friendly language. 


Internal taxonomy should not be external messaging. Not only does policy 
language not resonate with parents, it is often misinterpreted, which can 
perpetuate misinformation. 


Be cautious in assuming parents have a common understanding of ‘standard’ policy 
language. Common terms such as growth, proficiency, achievement gap and culture 
and climate are interpreted differently by parents. 


Only use policy language and technical terms when absolutely necessary, and provide 
clear definitions to avoid misinterpretation. 


Y Be transparent about the findings and thoughtful in how they are used. 


Upon the conclusion of all stakeholder engagement efforts, analyze the feedback 
to uncover the overwhelming and consistent themes. Determine how it impacts the 
development and language of your report card. Ask the following questions: 


o What specific information do parents want to see on the report card? What is 
possible and realistic for us to include? 
o Were there clear recommendations on how parents want to access the report card? 


o What feedback did we receive that we cannot realistically implement? 
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o What language resonates with parents? 


o What is parents’ current level of knowledge about report cards and 
accountability? What supplemental information will we need to provide? 


¢ Summarize and publicly share what you've learned and what decisions were made as 
a result of the learnings. Participants will need to see how their feedback was used 
in order to feel valued. If there was no clear agreement on a topic or question during 
discussion, be honest and discuss how a final decision was reached. Be explicit that 
the work is collaborative and not consensus-based, and therefore not all feedback 
could be implemented, but also be sure to share rationale for why feedback may not 
have been used. 


v View stakeholders as consumers and partners. 


¢ Do not think of this engagement as an isolated event, but as an opportunity to build 
sustainable relationships with parents, which can create more positive perceptions about 
the state’s education agency and a more vested interest in the changes taking place. 


¢ Continue to listen to parents. Identify additional opportunities for parent and family 
engagement to demonstrate that you value their input. 
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